The invention relates generally to frameworks, and more particularly to an event processing system for processing real time data and non-real time data while executing programmable tasks.
In today's information-rich environment, the efficient handling of massive volumes of data is important and challenging. Typically, this data may be provided in streams, with, in many instances, data values being generated in real-time, as events occur. For example, microsensors used in radio-frequency identification (RFID) in tracking and access applications can provide streaming data on locations of objects being tracked. As another example, data defining financial transactions may be provided in a stream as those transactions occur.
For many businesses the ability to operate on streaming data arriving in real-time can provide significant competitive advantage. For example, financial operations that are based on results of financial transactions may receive streams of data on trades as they occur. Moreover, responding to particular signals in the streaming data quickly is often a critical aspect of many applications. As an example, network monitoring systems used by government agencies to detect security threats need to detect and report events represented in streams of data collected through monitoring.
However, in most applications, processing of streamed data is performed by first storing the data in a database. The database could then be queried to retrieve the data for further processing and analysis. Therefore, analyzing the data in real-time is difficult, because of the limits imposed by database access time, particularly for streams with high data rates.
Therefore, there is a need for an integrated system that enables the use of real time and non-real time data in event processing systems while executing several software applications.